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ABSTRACT 



The purpose of this thesis is to trace and analyze the 
evolution of Reliability and Maintainability as engineering 
disciplines. Articles published in the open literature were 
used as the measurement indicator for developing the growth 
curves of the various branches within each discipline. The 
growth curves were analyzed to determine the present 
emphasis and to project future trends within each discipline. 
Analyses were conducted to determine the relative contribu- 
tions made to the growth patterns by private and public 
organizations such as the Department of Defense, Service 
Industries, etc. Elements of each discipline which indicate 
probable future developments have been identified. Where 
possible, the factors contributing to future growth have 
also been identified. Taxonomies have been developed which 
provide a structured classification system for the various 
elements within each discipline. The authors believe that 
the taxonomies, in conjunction with the growth curves, 
present a comprehensible analysis of the evolution of the 
Reliability and Maintainability Disciplines. 
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I. INTRODUCTION 



A. BACKGROUND 

The ability o£ a system to be operationally ready when 
demanded is a function of its design. In the early 1940 's, 
the designer's primary goal was to achieve an output that 
would satisfy a desired set of specific performance 
requirements. Major shifts have evolved in this thinking 
with the rapid advancements in technology since World War 
II. The emergence of supersonic aircraft, nuclear sub- 
marines, moon landers, planetary exploration, laser tech- 
nology, and complex weapon systems has highlighted the need 
for efficient and economic designs in terms of system 
effectiveness and cost. Systems now are required to be both 
reliable and maintainable in order to be cost-effective over 
their designed life. 

B. PURPOSE OF THESIS 

The primary purpose of this thesis is to trace and 
analyze the evolution of reliability and maintainability as 
engineering disciplines as evidenced by the published 
literature. Growth curves are analyzed to determine where 
the emphasis in reliability and maintainability has been 
over the past three decades, what the emphasis is today, 
and, finally, to project probable future discipline emphasis. 
Secondary objectives were development of a classification 
structure for each discipline, and development of a 
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substantial data base o£ reliability and maintainability 
documents easily retrieval for research purposes. 

C. METHOD OF RESEARCH 

An extensive literature search was performed to develop 
a factual analysis of the evolution of these engineering 
disciplines. Several field trips were conducted to inter- 
view people who are considered to be highly knowledgeable 
in reliability and maintainability in order to obtain their 
perceptions of the evolution of these fields and to solicit 
their critiques on the proposed classification structures. 
Such contributions provided valuable assistance in the 
early phases of the project. 
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II. EVOLUTION OF RELIABILITY AND MAINTAINABILITY 



A, OVERVIEW OF THE EMERGENCE OF RELIABILITY AND MAINTAINABILITY 
Literature published on reliability, maintainability, and 
quality control was researched to identify the emergence of 
these disciplines and to quantify their growth patterns. 

Quality control is included because some of the concepts 
applied in the early reliability writings were developed in 
the quality control field. The Cumulative Book Index was 
initially used for this purpose because it is a reasonably 
complete and comprehensive list of works published in the 
English language. Also, works listed in this reference must 
be copyrighted and must have the majority of the material 
devoted to the subject under which the work is listed. 

These three factors (English language, copyright, majority 
of work devoted to listed subject) made the index an ideal 
source for providing an indication of the growth pattern of 
the reliability and maintainability disciplines. The Index 
provided title, author, and number of pages for each work 
listed. Figure 1 shows the evolution of quality control, 
reliability, and maintainability as evidenced by books 
published in these disciplines. The indicator for growth 
used is the number of pages published in three year intervals 
beginning in 1944 and extending through 1975. It should be 
noted here that any conclusions reached using these curves 
should be treated with care since books have an inherent 
time lag between original manuscript preparation 
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Figure 1 Emphasis on R & M from Books 
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and publication of up to three years. These curves are 
presented only for the purpose of giving a qualitative 
feeling for the discipline growths rather than trying to 
pinpoint exact emergence of the disciplines. 

As Figure 1 indicates, books on quality control peaked 
about 1951 or 1952 and declined thereafter until 1965 when 
quality control was no longer listed as an entity in the 
Cumulative Book Index . Reliability began appearing in the 
early 1950's, and maintainability began appearing in the 
early 1960 's. Research into the periodical literature has 
provided a more detailed breakout of some of the events 
which have affected the development of these disciplines. 

B. GROWTH OF RELIABILITY AND MAINTAINABILITY BY THE DECADES 

1. The Years Before 1940 

Reliability principles were used in 1916 by the 
Western Electric Company which is the manufacturing unit of 
the Bell System. With production running at a rate of about 
ten million telephones annually, and rising rapidly, studies 
were initiated to discover means to produce trouble-free 
telephone equipment for public use. The Western Electric 
Company was among the first to realize that the statistical 
sampling methods already being formulated for science could 
be applied to industrial processes. The Bell System under- 
stood that durability must be a main goal, and that service 
history and optimization of the design for maximum quality 
were important factors. 
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2. 1940 - 1950 



The decade began with a national emergency in which 
there was an urgent need for development of a method for the 
manufacture of uniform high quality products. The need for 
greatly increased rates of production lent itself readily 
to the application of the new statistical techniques. Large 
quantity production in the rapidly growing electrical and 
electronic industries led the War Department to develop 
standards for the application of statistical methods to the 
quality control of materials and manufactured products. 

The rapid growth of the electronics industries also brought 
increasing problems of reliability. Radar and other military 
developments of World War II introduced the need for specific 
consideration of reliability. 

In 1946, the commercial airlines sponsored field 
studies of tube and electronic equipment failures. These 
studies were performed by Aeronautical Radio, Inc., (ARINC) . 

3. 1950 - 1960 

As a result of its work in developing reliable tubes 
for the airlines, ARINC was requested to make an investigation 
of military electronic tube reliability. From 1951 to 1954 
the program was concerned primarily with the evaluation and 
improvement of tube reliability. A general report summarizing 
the findings was published in 1954. The effects of applica- 
tion, environment, and operating and maintenance conditions 
were shown to be so closely related that the program was 
redirected to emphasize system reliability as affected by 
electron tubes. Systems under study expanded to radio 
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communication systems, radar systems, and bombing and 
navigation systems. 

The recognition by the Department of Defense that 
many parts other than vacuum tubes were causes for problems 
led to the formation of the Ad Hoc Group on Reliability of 
Electronics Equipment (AGREE) in 1952. This group was 
instrumental in initiating an increasing number of studies 
in order to add to their knowledge of equipment failures. 

The AGREE Committee was at least partially responsible for 
issuance of a Department of Defense Directive entitled 
"Reliability of Electronics Equipment" which required that 
additional emphasis be placed on the reliability of elec- 
tronic equipment. 

In 1950, the VITRO Corporation undertook a study of 
the reliability of Navy shipboard electronic components and 
equipments. This study resulted in the establishment of 
the relative failure rates of various parts and in the 
development of an improved failure reporting system. In the 
late 50 ’s many other companies made contributions to the 
improvement of military electronic equipment through field 
studies carried out under military contracts. These contri- 
butions were directed toward the measurement of equipment 
reliability and the development of methods for predicting 
the reliability of new electronic equipment while it was 
still in the design stage. 

The recognition of the equipment problems created a 
sense of reliability awareness during this period. Mass 
production of systems caused the need for standardized tests 
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to be used in the factory. Reliability standards had to be 
developed. Standardization of parts and circuits were 
stressed. Parts improvement programs were initiated as the 
quality of parts left much to be desired. Component 
specifications were now being written. The critical importance 
of reliability was recognized both in the Department of 
Defense and in industry. This importance was emphasized with 
issuance of Military Handbook 217 and Mil-Std-756A for 
Reliability Predictions as well as Mil-Std-785A for Reliability 
Program Plans and Mil-Std-781 for Reliability Demonstration 
Tests . 

Prior to 1954, maintainability was not a defined 
discipline. Some commercial manufacturers were incorporating 
features later to be known as maintainability features into 
the design of their products. An example of this was the 
production of standardized rifles (Ml and Carbine) for the 
U. S. Army during World War II. The U. S. Army required 
that the soldier be able to disassemble and assemble each 
acceptable rifle while blindfolded under combat and inclement 
weather conditions in mud. An additional requirement specified 
that the rifle must perform satisfactorily after such as 
assembly. Therefore, even though maintainability was not yet 
a widely accepted discipline, the Ml and Carbine rifles 
demonstrated the feasibility of planning for maintainability 
in the initial stages of program development. 

U. S. Government publications concerning maintaina- 
bility did not exist in this period. Maintainability 
requirements were covered through specialized contractual 
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exhibits and/or amendments to the contracts. By 1959, 
formalized program specifications started to evolve. The 
U. S. Air Force maintainability requirements for aerospace 
systems and equipment (MIL-M- 26512) was one of the first. 

4. 1960 - 1970 

This decade witnessed the rapid growth of the 
maintainability discipline. A realization that the best 
design from the reliability standpoint may be poor from 
the standpoint of maintainability created a new challenge 
for the design engineer. The need for maintainability was 
predicated on the basis that no system is totally reliable; 
i.e., the system will eventually fail and therefore considera- 
tions must be made on how to return the system to an opera- 
tional status as rapidly, as effectively, and as efficiently 
as possible. The awareness of the need to consider reliability 
and maintainability as design parameters early in system 
development began to evolve. 

The development of integrated circuits and their 
application during this period increased the complexity and 
sophistication of hardware. Worst case and statistical 
circuit analyses were used as design tools. Computerized 
failure history data banks for use in reliability predictions 
were developed. Reliability testing using statistically 
designed tests was introduced and became recognized as a valid 
test method. The systems effectiveness concept was extensively 
explored. In 1965 the first Naval Material Support Establish- 
ment Systems Performance Effectiveness Conference was held. 
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MIL-STD-470 was issued in 1966 as the Department o£ 
Defense preferred specification for Maintainability Programs. 

A separate ’’Maintainability Prediction” Handbook (MIL-HDBK-472) 
was issued as a companion to the new DOD maintainability 
standard. With issuance of a standard for maintainability 
demonstrations (MIL-STD-471) a fairly complete set of 
maintainability engineering implementation tools now existed. 

5. 1970 - 1976 

Cost factors became dominant during this period and 
emphasis was given to considerations such as life cycle cost 
and design-to-cost . Support costs became manpower intensive, 
as the average cost of a soldier doubled in these six years. 
Despite a reduction of defense personnel, operations and 
maintenance costs rose for fewer operational units, and new 
systems were being acquired at a slower rate. The extended 
time in service of the old systems increased reliability 
and maintainability problems which in turn increased the 
costs for operation and maintenance. 

Reliability methods and procedures developed in 
earlier periods were refined and were being extended into 
consumer, energy, and nuclear power areas. The airlines 
pioneered the MTBF guarantee which requires that the equip- 
ment supplier guarantee a stated mean- time-between-failure 
in the operating environment. If the guarantee is not met 
the supplier must provide corrective action and provide spares 
for use. This MTBF guarantee has been modified by the 
military, and DOD is using Reliability Improvement Warranties 
as an incentive for contractors to design equipment to meet 
specified reliability. 



In today's atmosphere of increased cost consciousness, 
there is continued emphasis on reliability and added emphasis 
on maintainability. The defense community emphasizes the 
total cost of ownership, the largest component of which is 
operating and support costs. This is leading to a search for 
innovative approaches to improve equipment maintainability. 
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III. METHODOLOGY 



A. INTRODUCTION 

It is apparent that reliability and maintainability have 
evolved from rudimentary concepts into full scale scientific 
disciplines over the past thirty years. To analyze this 
evolution, it was necessary to devise a means to measure 
their evolution and identify data sources applicable to the 
measurement technique. This chapter addresses the measure- 
ment technique, data sources, and validation procedures 
utilized. 

B. METHOD OF MEASUREMENT OF DISCIPLINE DEVELOPMENT 

It was desirable to employ a measure that would give as 
accurate a representation as possible across the full 
spectrum of both disciplines. Dimensions which were con- 
sidered in the selection of a measurement method included 
the impact of reliability and maintainability on industrial 
products both from user and producer viewpoints, growth 
measurement of such areas within the disciplines, applications, 
and available data base. It was considered important that 
the growth measure selected be broad enough to span all of 
the above considerations without a significant risk of data 
skewing. 

Several measures of discipline development were con- 
sidered and the measure ultimately chosen was the number of 
articles published in the open literature because this 
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measure provided the broadest coverage o£ the disciplines 
with the least amount of bias. Other measures, such as 
government specifications would tend to place undue emphasis 
on weapons systems and/or space related systems as opposed 
to consumer capital items or construction equipment. Focusing 
on commercial, industrial related measures would have de- 
emphasized reliability and maintainability impacts derived 
from the rapid state of the art advances in technology 
resulting from defense and space activities. Industries 
(particularly commercial products) tend to make small incre- 
mental changes in their product technology levels whereas 
DOD and NASA tend to make quantum jumps. In one way or 
another all of the alternatives bias the data. However, it 
appeared that tracking the open literature, such as annual 
reliability and maintainability symposia, on a yearly basis 
would provide the broadest indicator of discipline development. 

1 . Discipline Structure Development 

Once the method for measuring discipline development 
had been selected, it was necessary to address questions 
concerning the breakdown of each discipline into subelements. 
Initially it was unclear how to subdivide them and to what 
depth. The disciplines could, for example, be divided into 
functional and application oriented subelements or they 
could be subdivided by the physical and mathematical 
sciences forming the core of underlying theory. All of these 
subdivision alternatives had merit, and it was decided to 
incorporate them into a hierarchical classification system. 

As a result, a taxonomy was developed whereby articles could 
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be classified and the data stored for future analysis as 
well as providing a mechanism for article retrieval for 
research purposes. The taxonomy provided an excellent 
structure for analyzing the development of these disciplines 
along the dimensions mentioned earlier. It was particularly 
useful in defining the main branches and emerging subbranches 
of the scientific core of the disciplines. 

The keywords were initially selected by researching 
a representative sample of the available literature and 
through successive refinements were finally arranged into a 
classification structure. The final determination of key- 
word location in the taxonomy and the phrasing of keyword 
definitions was not a trivial effort. Consider, for example, 
whether prediction should merit its own elemental category, 
be included with statistics as a subelement under quanti- 
tative methods, or be included as a subelement under analysis. 
At one time or another in the development of the taxonomy, 
prediction was classified as all three. Eventually, it was 
agreed that prediction was most frequently utilized in 
practice as an analysis technique and it was positioned in 
the taxonomy accordingly. Similar iterations have been 
performed on most of the main branches of each discipline in 
arriving at the final structural relationships. 

The structure was then presented to several persons 
with extensive experience in the fields of reliability and 
maintainability for comments. This process was iterated 
several times and resulted in the taxonomies presented in 
Figure 2 (Reliability) and Figure 3 (Maintainability) . 
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2 . Keyword Definition Consistency 



Along with the determination of an appropriate 
classification structure, it was particularly important to 
maintain consistency in the manner in which articles were 
classified in the structure. For consistency, it was 
necessary for everyone involved in the study to have a clear 
understanding of keyword definitions within the context of 
discipline usage. There did not appear to be a universally 
accepted glossary of reliability and maintainability terms 
available from the literature. Where possible, generally 
accepted definitions were used. However, in some cases it 
was necessary to develop definitions either because none 
existed or because there were conflicting usages in the 
literature. Considerable effort was expended early in the 
study to converge on a set of keyword definitions and to 
solicit comments on the proposed definitions from experienced 
leaders in their respective disciplines. The definitions 
finally arrived at are included in Appendix A. 

Each article classified was studied to determine the 
author's intended thrust and appropriate keywords were chosen 
to describe the primary thrust at each level in the hierarchy. 
If secondary objectives were identified, these also were 
classified. However, it was considered important to determine 
the primary thrust first as this would insure that the articles 
were read in sufficient depth to minimize classification 
errors. The main keyword classification system presently in 
use in the literature has evolved from the ASQC system over 
a period of years without a great deal of planning either 
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with regard to future expansion or consistency with past 
evolution. It was also apparent that authors had classified 
their own works, and in many cases this led (in our opinion) 
to a substantial portion of the articles being classified 
with inappropriate keywords. 

Attempts were made to use the keywords already 
assigned to certain articles but this proved difficult and 
even misleading. In a few cases it was apparent that the 
author had attempted to expand the scope of his article by 
inappropriately expanding the keyword designators. It was 
felt that the lack of generally accepted keyword definitions 
combined with the need for a clearly defined and documented 
structural relationship of the keywords within the disci- 
plines greatly contributed to the article author's 
classification problems. Recognition of this situation and 
desire for measurement consistency contributed to the 
taxonomies depicted in Figure 2 and Figure 3. 

C. TAXONOMY CONSTRUCTION 

The taxonomies are arranged in a hierarchical order 
for the two engineering disciplines. The top 
three levels, functions, applications (general), and appli- 
cations (specific) were established to enable discipline 
growth to be measured along these dimensions as well as 
within the branches or elements of the discipline. In 
terms of growth measurement, it appeared worthwhile to 
provide a means for separating government-oriented applica- 
tions such as defense and space systems from industrial 
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Specifications 



applications such as process control. As shown in Chapter IV, 
it is interesting to correlate various functional activity- 
levels with elemental or branch developments. An example 
of this might be correlation of Department of Defense in- 
fluence on analysis of reliability in electronics systems. 

The levels of ultimate interest, of course, are the elements 
and subelemental development trends in both disciplines. 
Tracing of the development of these lower levels chronolog- 
ically provides a great deal of insight into how the disci- 
plines evolved to their present status, and correlation with 
the upper levels provides an insight into the underlying 
factors influencing their development. A secondary objective 
can be realized by retaining these upper levels because they 
greatly aid in document identification and retrieval for 
research purposes. 

As indicated in Figure 2 and Figure 3, the keyword 
structures for reliability and maintainability have a great 
deal of similarity particularly at the upper levels. This 
is not accidental nor is it inappropriate if one considers 
that the disciplines are heavily interdependent in terms of 
both application and functional dimensions. Emphasis and 
hence growth stimulus have varied greatly within the 
functional categories over the years. For example, govern- 
ment agencies such as the National Aeronautical and Space 
Administration (NASA) have been extremely interested in 
reliability whereas the Department of Defense (DOD) has 
emphasized both reliability and maintainability in a more 
balanced sense. 
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In attempting to visualize the development of these 
disciplines, it is helpful to compare their development to 
that of a tree as pictorially illustrated in Figure 4. One 
might represent the roots (which supply nutrients for growth) 
as the sciences of mathematics, chemistry, physics. These 
supply the technology and innovation for discipline develop- 
ment. The trunk represents the core of the disciplines 
such as theory of failure and theory of repair. The branches 
represent growth of elements and subelements. As the tree 
grows certain branches exhibit high growth rates and then 
tend to stabilize or even stagnate and die. These same 
characteristics are exhibited in both the reliability and 
maintainability discipline developments. Thus, the analogy 
is helpful in visualizing the growth patterns. 

In drawing this analogy it is noted that maintainability 
has developed along similar lines to reliability and in many 
ways, was an outgrowth of reliability. It should be noted 
that basic differences between the two disciplines are 
primarily due to different underlying concepts and theory. 

In the analysis which follows in Chapter IV, reliability 
and maintainability are treated as separate entities, and 
growth of branches and subbranches within each discipline is 
measured and analyzed. The similarities and differences 
referred to in the preceding discussion become more apparent 
as the measurement and analysis of their development from a 
chronological standpoint is discussed in detail. 
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D. DATA SOURCES 



In attempting to quantify the growth of the various 
branches within disciplines, it was decided to emphasize 
articles published in the open literature as opposed to 
books because articles were much narrower and more specific 
in scope. Books tend to be tutorial in nature, and in 
general, are not representative of the taxonomy. It was 
necessary to examine and classify a large number of articles 
in order to develop a data base which was large enough to 
be significant and which, in the aggregate, would have 
minimum bias. There is, of course, a much larger data base 
available and the challenge was to select those sources 
which would be most suitable for the purposes of this thesis. 
It was desirable to include articles from sources which were 
somewhat continuous in nature and which, in the aggregate, 
covered the broad spectrum of both disciplines. It was 
important to ensure that a concentration of articles 
covering a narrow spectrum of the disciplines were not in- 
corporated in any given time interval. Otherwise, conclusions 
based on the sample results would not be representative of 
the total population. In this regard, reliability and main- 
tainability symposia proceedings were selected as the 
primary sources for articles since they are continuous in 
nature and tend to cover a broad spectrum of topics. Several 
different symposia were chosen since each one tends to em- 
phasize different branches of the disciplines, and together 
they appear to provide a well-rounded coverage of each 
discipline . 
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Time phasing o£ these sources and their respective mergers, 
terminations, etc., is shown in Figure 5. For reasons of 
simplicity, as symposia and proceeding titles changed fre- 
quently, the most recent symposia proceedings titles were 
used. A complete list of article sources chosen for 
analysis of discipline growth is presented in the List of 
References. The reference numbers in the figure correspond 
to the identifier numbers in the List of References. As 
indicated by Figure 5, the sources selected span the time 
period of interest with reasonable consistency and, in 
aggregate, represent approximately 4,000 articles. Two 
seemingly reasonable sources were not included in the data 
base. The Proceedings of the West Coast Reliability Symposium 
were not available. The Reliability Abstracts and Technical 
Reviews published by NASA between 1960 and 1970, did not 
appear to have reliability as a major thrust (abstracts of 
various reliability symposia excepted). In addition, they 
were of such brevity to make classification to the desired 
depth extremely difficult, thus greatly increasing the risk 
of erroneous classification. 

Figure 6 indicates the relative contribution each of 
the sources made to the overall data base used for intra- 
discipline growth analysis. The data base for reliability 
was reasonably continuous and uniform in density. The data 
base for maintainability, however, was very spotty with 
conferences lasting only a few years before termination. 
Maintainability is hard to find in the open literature after 
1970. 
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Figure 5 Literature Sources 
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Figure 6 R & M Data Bases 
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It was decided that the Proceedings of the Annual 
Reliability and Maintainability Symposium would be an 
excellent base for developing the taxonomy. These pro- 
ceedings started in’ 1954, and, with the exception of 1955, 
have been published annually since. The reasons for this 
decision were multiple: 

1. The source was readily available. 

2. The proceedings present a continuous and long time 
span of articles in the reliability discipline. 

3. The proceedings are a representative and compre- 
hensive collection of articles encompassing the range of 
skills within the disciplines. 

From initial readings of the proceedings , it became 
apparent that the taxonomy for reliability could readily be 
constructed from the article content. 

The taxonomy for maintainability, however, could not be 
constructed from this source because of an insufficient 
quantity of articles. The proceedings were not officially 
designated a maintainability source until 1972 ;vhen the title 
changed from Proceedings of the Annual Symposium on Reliability 
to Proceedings of the Annual Reliability and Maintainability 
Symposium . This occurred as a result of the merger of the 
Aerospace Reliability and Maintainability Symposium with the 
Annual Symposium on Reliability as indicated by the arrow in 
Figure 5. Other sources, therefore, were investigated for 
development of the taxonomy for maintainability. 

The Maintainability Committee of the Electronics Industries 
Association (EIA) compiled two volumes of a maintainability 
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bibliography, with the first covering the period to 1964 and 
the second covering the period 1964-1968. These two volumes 
represent the first decade of active development of the 
maintainability discipline. A close scrutiny of this 
reference was made in an attempt to locate a source or sources 
that would offer the same advantages for research as the 
Annual Reliability and Maintainability Symposium Proceedings. 
An initial source selected was the Proceedings of the NMSE 
Systems Performance Effectiveness Conference (SPECON ) which 
was in existence from 1965 to 1969. This source was pri- 
marily concerned with qualitative aspects and was deficient 
with regard to the quantitative aspects of maintainability. 

A second source was selected, Proceedings of the EIA 
Conference on Maintainability of Electronic Equipment , which 
covered the years 1957 to 1963. Now, with the Annual 
Reliability and Maintainability Symposium Proceedings , an 
overview of Maintainability existed for the period 1957 to 
1976. It is from these sources that the taxonomy for 
Maintainability was established. 

E. TAXONOMY VALIDATION 

Throughout the development of the taxonomy, the classi- 
fication scheme was subjected to a series of checks, primarily 
relating to the naturalness of the keyword groupings and 
their relation to the scientific basis for the underlying 
theory. The initial groupings were taken from the American 
Society of Quality Control (ASOC) classification system and 
were subsequently refined and modified as more insight was 
gained through research of the literature. As the taxonomies 
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evolved, many ambiguities and inconsistencies surfaced which 
required resolution. This was accomplished by conversations 
with experienced practitioners in the fields of reliability 
and maintainability and by a trial run consisting of classi- 
fying a large number of articles to expose the broad 
spectrum of subbranches within the disciplines. As ex- 
perience and depth of knowledge about the disciplines increased 
through exposure to the literature, it became progressively 
easier to resolve the ambiguities. 

In a less formal, yet equally meaningful sense, the 
taxonomy was validated when articles began to be classified 
with relative ease. Finally, a classification scheme was 
developed that corresponded to the opinions of the practi- 
tioners and which appeared to fit the patterns established 
by an analysis of the literature content. 

F. DATA STORAGE AND RETRIEVAL 

It was apparent from the beginning of the thesis research 
that, because of the large number of articles to be read and 
classified, there would be a monumental task associated with 
the storage, control, and manipulation of the data. To 
maintain control, each article was assigned a series of 
codes which served to distinguish it from all others. An 
IBM 360/67 computer was used to store and manipulate the 
generated data. This allowed for data storage by article 
title, author, date of publication, publisher, keyword, and 
combinations thereof. Specific information regarding the 
assignment of keyword codes, data update procedures and 
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related computer programs is included in a User's Manual 
written as a companion document to the thesis.^ Each 
article entered into the data bank was analyzed for content, 
and classified by keyword according to the taxonomy developed 
and presented earlier. This data (number of articles by 
keywords) was then recalled, totaled by year for each key- 
word, and plotted in the form of a histogram. The histograms 
provided considerable insight into significant changes in 
emphasis which occurred in the time period of interest. The 
data was then summed and plotted in a cumulative form to 
gain a perspective of the overall growth characteristics of 
each branch. This also provided a good indication of branches 
which had matured or which were in the process of maturing. 



^Hasten, R. L. , Hamilton, T. A., and DiPasquale, J. A., 
User's Manual for "A Study of the Evolution of the Reliability 
and Maintainability Engineering Disciplines" Computer Programs , 
Naval Postgraduate School, 1977 . 
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IV. ANALYSIS OF RELIABILITY AND MAINTAINABILITY 
DISCIPLINE EVOLUTION 



A. INTRODUCTION 

This chapter addresses the evolution of the reliability 
and maintainability disciplines from a chronological stand- 
point. The measurement technique discussed in Chapter III 
is used to analyze the development of the disciplines along 
the branches and subbranches depicted in the taxonomies. 

Each discipline is analyzed as a separate entity in terms 
of its internal development. Similarities and differences 
in the development of the two disciplines are noted in the 
concluding section of this chapter. 

B. RELIABILITY DISCIPLINE GROWTH 

The overall trend of the reliability discipline is 
discussed before analyzing its intradisciplinary growth. 
Figure 1 presented in Chapter II provides one measure of the 
overall discipline growth using number of pages published 
in book form as the measure of growth. Books, however, do 
not present the whole picture because of the time lags 
associated with publishing large works and because of the 
wealth of literature published in short article form. To 
gain a perspective of the overall discipline growth a 
composite picture \<as formed using both books and short 
articles. Figure 7 presents this composite overview, and 
again the unit of measure is number of pages published in a 
three year interval . This unit of measure is necessary in 
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No. of Pages Published in 3 year interval 



Figure 7 R Discipline Emphasis 
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order to directly compare books and articles, and it also 
provides a smoothing function so that the long term trend 
is discernible. 

Figure 7 indicates that interest in reliability gradually 
increased from the early 1950 *s to about 1960 and then 
dramatically increased until about 1970. After 1970 there 
appears to be a gradual decline in growth. Reasons underlying 
this phenomenon can only be conjectured at this point. The 
growth during the 1960 's can be largely attributed to the 
intense interest of the National Aeronautics and Space 
Agency (NASA) and the Department of Defense (DOD) and the 
accompanying financial resources which stimulated the 
aerospace industry in this time period. The decline noted 
during the early 1970 's could be the sign of a maturing 
discipline, or it could be due to economic factors, since 
both space and defense have suffered budget cut-backs 
(especially in research and development) in this time frame. 
The decline is probably a combination of the above factors. 
However, the economic constraints are likely to be pre- 
dominant because technological advances which allow 
development of increasingly complex systems stimulate 
technical innovation from reliability engineers. 

C. RELIABILITY INTRADISCIPLINE GROWTH 

The method of analysis utilized to determine the growth 
characteristics of the branches and subbranches of each 
discipline has been discussed in Chapter III. For the 
periodical literature, the measurement indicator is number 



41 



o£ articles published per year, and is plotted in the form 
of cumulative histograms for each of the branches and twigs 
identified by keywords. The branch, subbranch, and twig (us- 
ing the tree analogy) growth patterns will be investigated 
prior to correlating these elements with the upper divisions 
in the taxonomy. 

1 . Reliability Branch Growth Trends 

. Analysis of the main branches of the taxonomy (theory, 
design, etc.) was performed utilizing the cumulative number 
of articles published during the time interval of interest. 
This is a measure of the interest and resources which have 
been applied to the various branches of the discipline over 
the past twenty-five years, and represents the accumulation 
of knowledge within each branch. This data is graphically 
presented in Figure 8 for the period from 1950 to 1976. 

The amplitudes of the curves provide a good indica- 
tion of the relative attention paid to the various branches. 
The curves indicate that management, analysis, test and 
evaluation, and design received far more emphasis than did 
theory and data. The slope of the curves indicates the 
emphasis that each branch received at a particular instant 
in time. 

Theory and data (which includes systems and tech- 
niques for collection, processing and utilization of the data) 
appear to have small and reasonably constant growth except 
for a slight increase in emphasis in 1975 and 1976. The 
other four branches indicate a substantial increase in 
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Figure 8 R Branches 
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Year 



emphasis in the 1960 *s and 1970's. Analysis exhibited the 
most dramatic change, primarily due to the influence of 
NASA and DOD. 

The starting points for the various curves show 
when the literature began to emphasize each branch, and 
do not imply that there was nothing written previously. 
Management emerged early in the late 1940 's and early 1950 's, 
providing the transition from the quality control era into 
the reliability era. This phenomenon was also noted in books 
as those works slowly transitioned from emphasis on quan- 
titive aspects such as statistical sampling theory to 
qualitive aspects of process control, and management structures 
for quality control. Theory apparently emerged (late 1950 *s) 
shortly after design and analysis. This seems reasonable, 
since in the process of design for fewer failures, and in 
the analysis of failures, engineers and scientists probed 
deeper into the underlying physical causes of failure in 
devices . 

These curves provide an overview of the growth 
patterns of the main branches of the discipline and it 
appears that none of them have reached maturity. Analysis 
and Management show strong tendencies toward increased 
emphasis in the immediate future. Within each main branch 
there are several subbranches which also have pronounced 
growth patterns and these are discussed in the following 
sections . 
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2 . Analysis Branch 



The literature indicates that the Analysis branch 
can be split into three subbranches: (1) Configuration, 

which consists of those analytical techniques dealing with 
the physical composition of systems of subsystems. (2) Pre- 
diction, and (3) Reliability growth. The cumulative growth 
patterns exhibited in Figure 9 indicate the relative weights 
given the subtopics in the literature. It is noted that 
configuration which includes several analysis techniques 
such as modeling. Failure Modes Effects Analysis (FMEA) , 

Fault Trees, and Reliability Block Diagrams emerged in the 
mid to late 1950 ’s. It received a large growth stimulus in 
the early 1960 's and then settled into a fairly stable 
growth pattern which has continued until the present. The 
early stimulus was primarily due to interest from DOD and 
NASA, with the power (energy) and the service industries 
picking up the interest in the 1970 ’s. 

Prediction has exhibited characteristics similar 
to configuration but with somewhat lower emphasis. It too 
appears to have received increased emphasis in the mid 1970 ’s 
primarily from DOD and the service industry. 

Reliability growth has not shown significant develop- 
ment in comparison to the other subbranches. There has been 
a small and relatively consistent number of articles each 
year which have addressed this subject. Most of the articles 
seem to be concerned with trying to show mathematically 
whether a product will achieve, or has in fact achieved, a 
specified reliability level. 
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Figure 9 1^ Analysis Branch 
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H Growth 



Both the Configuration and Prediction subbranches 
have been further subdivided into twigs. The twigs which 
have grown out of Configuration are illustrated in Figure 
10. Modeling/Simulation and Failure Analysis experienced 
an extremely sharp increase in growth rate in 1962 and 
this appears to be primarily due to interest from DOD. The 
other areas appear to have experienced a reasonably con- 
sistent growth pattern. However, Design Review as an 
analysis technique appears to have matured in the late 
1960's. Fault Tree Analysis appears to be receiving in- 
creased emphasis in the 1970's. This could be caused by 
the growing interest in safety, stimulated by the electrical 
power industry. The technique is presently one of the strong 
bridges between reliability and safety analysis. 

The literature indicates Prediction split into two 
major areas: Parameter Prediction (estimation) and 

Probability Distributions. Each of these twigs received 
impetus about 1960 (see Figure 11) which resulted in an 
increased growth rate. Parameter Prediction has sustained 
the increased growth rate occurring in the early 1960 's. 
However, there is indication of a decrease in growth of 
Probability Distributions after 1965 in comparison with the 
other twig. Maturity has not yet occurred, although the 
trend indicates that it is probably not far off. 

The third area under Prediction, Apportionment, 
emerged in 1961, flourished for several years, and then 
appears to have died off in the late 1960 's. 
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Figure 10 Configuration Subbranch /Modeling/Simulation 
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Design Review 
Criticality Analysis 
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Figure 11 Prediction Subbranch 
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195S 1960 1965 1970 1975 



3 . Management Branch 



This branch is very broad in scope and is the most 
complex to discuss since numerous subbranches and twigs have 
emerged since the 1950 's. Figure 12 depicts the structural 
development o£ this branch using the tree analogy. 

Figure 12 is intended to display the structural 
relationships o£ the Management branch without trying to 
introduce the time dimension into the illustration. 

The major subbranches o£ Management are illustrated 
in Figure 13. All o£ the subbranches and twigs, with the 
exception o£ Product Liability, have received major impetus 
to their growth £rom DOD and NASA. 

The Reliability Program Management, Cost, Failure 
Recurrence Control, and Procurement subbranches all emerged 
very early in the literature. As noted in Figure 13, 
Reliability Program Management has received £ar more emphasis 
than any o£ the other subbranches. This subbranch received 
a major impetus £rom DOD in the early 1960 's and the in- 
creased growth pattern has carried on into the early 1970 's. 
There are indications that it may be approaching maturity 
because o£ the economic cut-backs mentioned in Chapter II. 

The other main subbranches. Failure Recurrence Control, 
Procurement, and Cost appear to be exhibiting £aijly con- 
sistent growth patterns. Failure Recurrence Control (e££orts 
by Management to curtail and/or preclude recurrence o£ 
reliability Failures) is indicating signs o£ approaching 
maturity in the mid-1970's, whereas both Procurement and 
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Figure 12 R Management Branch Structure 




51 



600 X Figure 13 Management Branch 
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Cost are showing signs of increasing emphasis. This is 
especially true of the twig, Systems Effectiveness, shown 
in Figure 14. 

Systems Effectiveness emerged in the early 1960 *s 
and has established and maintained a very healthy growth 
pattern to the present. This area has been primarily 
driven by DOD, and it is expected that this emphasis will 
continue, at least in the foreseeable future. Product 
Liability emerged in the early 1970 *s and is giving every 
sign of growing into a major area in the near future. 

The main subbranch. Reliability Program Management, 
has developed several twigs, and their growth patterns are 
illustrated in Figure 15. All of these areas have sustained 
fairly consistent growth patterns with the exception of 
Training/Education which appears to have reached maturity 
in the mid 1970*s. Field Activity emerged in the mid-1960's 
and has only recently experienced some growth. 

Evaluation and Assessment can be further split into 
three twigs as shown in Figure 16. It can be noted that 
all three of these areas have received added emphasis in 
the early 1970’s. 

Procurement, the remaining subbranch of Management 
to be discussed, has generated three major twigs as illus- 
trated in Figure 17. The growth patterns for specifications 
and contracts exhibit somewhat similar characteristics with 
both having received stimulus in the mid 1960 's and much 
less emphasis in the early 1970 's. Data from the last couple 
of years, however, indicates renewed emphasis in these areas 
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Figure 14 System Effectiveness 
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Figure 15 Program Management 
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Figure 16 Evaluation & Assessm3nt Subbraneh 
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Figure 17 Procurement Subbranch 
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primarily from DOD and the service (utilities, airlines, 
etc.) industries. Warranties emerged in the late 1950 's 
and continues to exhibit a pattern of increased growth. 

4 . Test and Evaluation Branch 

The Test and Evaluation (T § E) branch, illustrated 
in Figure 18, is split into three main subbranches: 

Methods (of testing), Statistics, and Reporting/Evaluation. 
There appears to have been greater emphasis given to Methods 
than either of the other two subbranches. This emphasis 
seems to have occurred in the early 1960 ’s and the resulting 
growth pattern has continued on into the present. The T 8 E 
branch emerged in the earliest literature and appeared to 
be an extension of the Quality Control (QC) discipline. The 
literature gradually transitioned from Quality Control 
related issues into Reliability during the 1950 's. The 
combined influence of DOD and NASA appear to be the major 
motivation for the rapid increase noted for Methods of 
Testing during the early 1960 's. 

Statistics was definitely a carry over from Quality 
Control and development of several twigs in this subbranch 
are noted in the literature. These are illustrated in 
Figure 19. Sampling Plans and Design of Experiments are 
the most direct carry overs from Quality Control and it 
appears that Design of Experiments has matured in the mid 
1970's. Bayesian techniques and Parameter Estimation have 
exhibited the most active growth patterns. Parameter 
Estimation had exhibited a relatively mild growth rate until 
1970, at which point it appears to have received added 
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emphasis. This seems to correlate with the increased 
attention from the service industries, especially the Power 
Industry, to testing in general. 

Bayesian techniques emerged about 1963 and exhibited 
a rapidly increasing growth pattern until about 1972 when 
it dramatically decreased, but seems to be picking up again. 
The reasons underlying this are not identified at this 
point. The other twigs. Confidence Level and Goodness of 
Fit, have exhibited low growth characteristics, and there 
is insufficient data available to comfortably project future 
activity in these areas. 

5 . Design Branch 

This branch has produced several twigs which, for 
classification purposes, have been treated as separate 
entities. Design has been split into component related 
activities and engineering analysis activities to separate 
the curves for ease and clarity of data presentation. This 
data is presented in Figure 20. From the upper portion of 
Figure 20, it can be noted that Part/Material Selection and 
Component Testing have been the most active areas in this 
branch. Both of these twigs exhibited dramatic increases 
in groii/th starting about 1964, and the increased growth rate 
has continued to the present. The sharp increase in growth 
appears to have been initiated by interest from NASA and 
later sustained by both DOD and NASA. The other two areas. 
Redundancy and High Reliability Parts, have exhibited much 
lower growth patterns. In fact. High Reliability Parts 
appears to have reached maturity. Redundancy appears to 
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be receiving increased emphasis in the mid 1970 's and this 
seems to be primarily due to articles concerned with 
reliability in the Mars Probe and Space Shuttle (space 
applications) . 

The other twigs which have developed from the Design 
Branch are presented in the lower portion of Figure 20. 
Criteria/Requirements and Tolerance/Safety factors have 
exhibited reasonably consistent growth patterns since they 
emerged in the mid 1950’s. Tolerance/Safety factors re- 
ceived a surge of interest in the mid 1960 's but soon 
settled back into its previously established growth pattern. 
Test equipment emerged in 1960 and appears to have received 
consistent emphasis until the mid 1970 ’s at which point it 
seems to have matured. Perhaps it is being presented in 
other formats. There has been relatively consistent dis- 
cussion in the literature of burn-in since about 1965 to the 
present. 

6 . Theory Branch 

This branch discernibly emerged in the literature 
in the mid to late 1950' s. The primary focus appears to be 
Theory of Failure and this subbranch has exhibited a 
relatively constant growth pattern since the sharp increase 
in slope in about 1961, (see Figure 21). Renewal Theory as 
applied in this context, has emerged as a twig in the early 
1960's and has exhibited a fairly consistent growth pattern 
to /the present. Renewal theory, as treated in the literature, 
is another of the bridges between reliability and maintain- 
ability. 
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7. Data Branch 



The Data Branch has received the least emphasis of 
all the main branches of the reliability discipline. 

However, it has demonstrated a reasonably consistent growth 
pattern, as shown in Figure 22. This branch has produced 
two major twigs. Data Collection and Data Processing/ 

Utilization. Both twigs emerged very early in the 1950 *s 
and exhibited a fairly constant growth pattern until 1969 
or 1970 at which point the growth curves show an increasing 
slope. This trend seems to be resulting from increased 
emphasis by both DOD and the service industries on the sub- 
ject of making effective use of data for both reliability 
and maintainability analysis, prediction, allocation, and 
spare parts determination. 

8 . Reliability Branch Correlation with Specific Applications 
Articles which could be classified along the applica- 
tion dimension as well as along the reliability intradisci- 
pline dimensions have been retained in the computer stored 
data base and provisions were made to retrieve these articles 
by requesting the appropriate combination of keywords. This 
has provided a good mechanism for correlating the growth 
of the more active branches with specific applications of 
the techniques contained within the branches. Only the four 
most active branches (Design, Analysis, Management, and 
Test) have been selected for correlation because these re- 
flect the major emphasis of the reliability discipline and, 
hence, contain sufficient data to establish some meaningful 
trends. Figure 23 presents the correlations of Design and 
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Analysis with specific applications. Figure 24 presents 
Management and Test methods also correlated with specific 
applications . 

In Figure 23, the curves correlating Design with 
specific applications indicate some rather dramatic changes 
in emphasis of this branch over the last two decades. 

■ Design applied to Space Transportation has received more 
emphasis than any other application during the 1960 's and 
* then it appears to have matured in the 1970’s. Air 

Transportation design applications were mentioned in the 
earliest literature, and it has displayed a relatively 
constant growth rate until the mid 1970 's when it seemed 
to approach maturity. Communication and Medical Applica- 
tions also seem to be approaching maturity in the mid 1970 's 
after having experienced fairly heavy emphasis in the late 
1960 's and early 1970 *s. Computer Applications (including 
both software and hardware) emerged in the literature in 
the late 1950 ’s and has experienced increasing emphasis 
ever since. Ground Transportation (primarily rapid transit) 
emerged in the mid 1960 *s and is experiencing increasing 
emphasis in the mid 1970’ s. Power Generation (includes both 
nuclear and conventional) emerged in the early 1970 's and is 
showing signs of phenomenal growth during the 1970 's if 
the present trend continues. 

The lower set of curves in Figure 23 indicates the 
relative emphasis which Analysis has had in various applica- 
tions. The same general trends are evident in this branch 
as were noted for the Design branch. However, it appears 
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Figure 24 Correlation of T & E and Management with Specific Applications 
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that Analysis has lagged Design by two or three years. 

There does not appear to be any Analysis applications that 
have matured, although Space and Air Transportation are 
exhibiting signs that this may happen in the not too dis- 
tant future. Analysis applications emphasis, like Design, 
appears to have shifted to Computer and Power Generation 
in the 1970's. Both of these application areas have re- 
ceived very high growth stimulation in the last few years 
and there is every reason to believe that these trends 
will continue, at least in the near future. 

Figure 24 presents the Test S Evaluation and 
Management branches correlated with the same specific 
application areas discussed earlier for the Design and 
Analysis branches. Some of the trends established for 
Analysis and Design are also evident in Figure 24. For 
example. Space Transportation applications has clearly 
reached maturity in the early 1970 's for both T § E and 
Management. Other applications presented in the T § E 
portion of Figure 24 all indicate stable growth patterns in 
the 1970 's. T S E application in the Power Generation 
field emerged in the mid 1970 's and is in such an infant 
state that trend prediction will not be attempted now 
except that, it is likely that it will experience signifi- 
cant growth because of the increase in the other branches. 

Management applications other than Space 
Transportation and Communications appear to be exhibiting 
strong, stable growth patterns. Ground Transportation 
emerged in the mid 1960 's and has achieved a strong growth 
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pattern (most of the emphasis here has been in rapid transit 
considerations for cities) . Power Generation has again 
taken off with every indication of establishing a very strong 
growth pattern. 

Management applications in the Medical Field emerged 
in the late 1960 's and appears to have established a stable 
growth pattern. A large part of the emphasis here has 
dealt with the effect of medical device failures on patient 
safety. 

9 . Reliability Branch Correlations with General Applications 

Throughout the preceding discussions of the evolu- 
tion of the reliability discipline, several statements have 
been made concerning factors and organizations which have 
provided major stimulus to reliability growth. Data which 
support these statements are presented in Figure 25 and 
Figure 26. These data are presented in the form of densi- 
ties instead of the cumulative format, in order to gain an 
insight into the emphasis that particular organizations 
have placed on the major Reliability branches at any given 
point in time. 

Figure 25 indicates the correlation of the Analysis 
and Design branches with selected general application 
categories. The top curve (double thickness) in each plot 
represents the overall density of articles which addressed 
Analysis or Design regardless of application. The cross 
hatched areas under these curves represent the emphasis 
placed by various organizations at various points in time. 

For example, in the Analysis portion of Figure 25, it can 
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be noted that DOD's emphasis on Analysis peaked in the 
mid 1960's and declined thereafter until the mid 1970's. 
Service industries (which include airlines, utilities, 
telephone companies, etc.) began emphasizing Analysis in 
the mid 1960 's and have progressively increased their 
interest to the present. ERDA began emphasizing Analysis 
in the early 1970's, also with steadily increased emphasis. 
The trends noted for Analysis are typical of those es- 
tablished by the other main branches. There are minor 
shifts in time when various branches were emphasized by 
either DOD or NASA, however the general trends appear 
consistent for the four branches analyzed in Figures 25 
and 26. 

Correlations of these branches were also run with 
the other general application areas indicated in the 
taxonomy. There was very low activity in all of the other 
areas with the exception of consumer capital items. This 
application has received steadily increasing emphasis in 
both analysis and design activity for the last two years. 

The literature surveyed does indicate, though, that DOD 
applications are beginning to place renewed emphasis on 
Management in the mid 1970 ’s. The literature indicates 
that this is probably due to increased awareness of the 
magnitude of their Operations and Maintenance Budget and 
the impact that reliability of equipment in the field has 
on this budgetary item. These same considerations have 
also provided a good deal of impetus to the growth of the 
maintainability discipline as described in the next section. 
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D. MAINTAINABILITY DISCIPLINE GROWTH 



A composite picture using both books and short articles 
was formed in order to gain a perspective of the overall 
discipline growth. Figure 27 represents- this composite 
overview. Again, the unit of measure for this comparison 
is the number of pages published in a three year interval. 

Figure 27 indicates that interest in maintainability 
began about 1957, sharply increased to about 1966, and 
then dramatically decreased until about 1972. The decline 
noted in the 1970 *s could be the beginning sign of a 
maturing discipline or it could be due to a shift in in- 
terest away from maintainability towards greater emphasis 
on reliability. The growth in complexity of systems led 
to the realization that equipment reliability could not be 
improved to the extent that the need for maintenance could 
be economically eliminated. Although the search for 
further reliability improvement continued, it became unwise 
to plan future systems without considering maintainability 
in the design. The significant increases in operating and 
support costs provided additional support for the rising 
interest in maintainability. This increase in costs occurred 
while developing agencies were working with increasingly 
limited funds. The overall growth of Maintainability was 
strongly influenced by recognition of this engineering 
discipline as a critical element in system effectiveness by 
the military. 
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E. NtAINTAINABILITY INTRADISCIPLINE GROWTH 



This analysis was conducted by plotting the cumulative 
number o£ articles published on selected topics o£ main- 
tainability as described earlier. 

1 . Maintainability Branch Growth Trends 

The curves in Figure 28 indicate that management 
has received £ar more emphasis than any o£ the other 
branches. This emphasis in management stems £rom the im- 
position o£ £irm military standards on all DOD contracts 
beginning in the late 1960 's. Analysis and Design emerged 
in the 1950's when maintainability criteria were being 
developed. Their growth then increased as maintainability 
requirements became better de£ined in the early 1960 *s. 
Analysis and Design seem to have matured in the 1970 's. 

Data growth rose in the early 1960 's, began 
£lattening out in the late 1960 's and appears to have 
matured in the 1970's. The rise appears to be primarily 
due to the military standard that directs the contractor 
to establish data collection, analysis, and corrective 
action systems. 

Maintainability Test 8 Evaluation emerged in the 
late 1950's, rose slowly in the middle 1960's and then has 
increased to the present. The growth increase noted in the 
late 1960 's appears to be derived £rom the military re- 
quiring maintainability demonstration tests by the contractor. 
Growth tendencies within each o£ the above main branches are 
developed in the £ollowing sections. 
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Figure 28 JM Branches 
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2 . Analysis Branch 



The literature indicates that the analysis branch 
can be split into Simulation/Modeling, Prediction, and 
Maintenance Engineering Analysis (MEA) . The growth 
patterns exhibited in Figure 29 indicate the emphasis 
given the subbranches in the literature. It can be noted 
that prediction received a large growth stimulus in the 
early 1960 's and then slowed down. This stimulus was 
primarily due to interest from the military services in 
developing methods of quantifying measures of maintaina- 
bility. Simulation/Modeling has exhibited the largest 
growth in this branch as engineers attempted to develop 
models to test and evaluate their assumptions. This 
subbranch growth increased from the middle to late 1960 's 
and then appears to have settled into a stable (and 
rapidly increasing^ growth pattern in the 1970 's. Main- 
tenance Engineering Analysis (MEA) has also shown a strong 
emphasis in the 1960 's with a lower, but steady growth in 
the 1970’s. The MEA growth in the 1960's can be linked 
to the added interest in Integrated Logistic Support (ILS) 
concepts. Maintainability Design Reviews as an analytical 
tool seems to have received some impetus in the mid-1960's 
and then matured in the early 1970 's. 

3 . Design Branch 

This branch has produced several twigs and Figure 30 
shows the relative emphasis each has received. Throughout 
the literature, human factors engineers have been described 
as those primarily responsible for the development of 
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Figure 29 M Analysis Branch / Simulation/Modeling 
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Design Reviews 



maintainability design guides. The curves indicate this 
interest began in the 1950 's and continued through the 
following decade. The declining growth in the 1970 's 
seems to reflect the maturation of ideas by the human 
factors engineers. 

The literature shows a rising interest in qualita- 
tive attributes from 1960 through the mid-1960's. The 
growth leveled off in the 1970 's indicating that qualitative 
design features had matured. 

Quantitative maintainability design criteria relate 
to equipment features that enhance maintenance time re- 
duction. These requirements have been specified for defense 
electronic, aircraft, and missile systems for about 15 years 
The rapid growth of design for quantitative requirements is 
graphically portrayed in Figure 30. The early quantitative 
maintainability specifications, established in 1960, 
stimulated defense contractors to put greater emphasis on 
design requirements. The effort to meaningfully quantify 
requirements continues to the present day. 

Trade-off studies as a design technique sustained 
a high growth rate in the 1960 's and then appears to have 
matured in the early 1970 's. 

4 . Test and Evaluation Branch 

The Test and Evaluation branch can be split into 
three main subbranches: test methods, statistics, and 

reporting/evaluation as shown in Figure 31. Greater 
emphasis appears to have been given to test methods than to 
either of the other two subbranches. This emphasis started 
in the early 1960 's and has continued to the present. 
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Figure 31 M Test & Evaluation ^lethods 
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The literature clearly indicates that DOD's interest 
in maintainability testing started in the early 1960*s. 

This interest has continued into the 1970 's with publica- 
tion of the updated Mil-Std-471A for maintainability demon- 
stration in March, 1973. This standard established uniform 
procedures, test methods, and verification for evaluation 
of the achievement of specified maintainability requirements. 

A normal carry-over from demonstration testing is 
the need for feedback data and other reporting mechanisms. 
Therefore, the reporting and evaluation subbranch received 
emphasis in 1965 and leveled off in the early 1970's. 
Additional growth has been stimulated in the mid-1970's. 

The application of statistics in developing test 
methods has been a natural development. Maintainability 
demonstration, for example, is primarily concerned with the 
measurement of active maintenance downtime, and measurement 
of both preventive and corrective maintenance downtime is 
generally conducted as a statistical test under carefully 
defined conditions. Mil-Std-471A emphasizes statistical 
techniques in its specified test methods, test parameters 
(mean, median, critical percentile), assumed distributions, 
sample size, sample selection, and specification require- 
ments. Interest in statistical techniques shows a steady 
increase starting in 1965 and maturing about 1970. 

5 . Data Branch 

Data collection and its utilization received 
significant interest in the 1960 's (see Figure 32). Efforts 
to develop maintainability data collection systems in 
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Figure 32 M Data Branch 
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response to military requirements provided a great deal of 
this impetus. Data collection grew dramatically in the 
1960 ’s and appears to have matured in the 1970 's as the 
military services implemented the developed systems. The 
Navy 3M system is an example of such a data system. Data 
utilization, however, continued some growth in the 1970 's 
and appears to be growing still. This growth can partially 
be attributed to the increased uses to which these data 
are being put; for example, quality assurance feedback, 
product improvement requirements, and spare parts manage- 
ment. The literature also indicated interest exhibited 
in a technique called Design Disclosure formats. This 
method of disclosing design information emerged in the 
early 1960 's and appears to have matured in the 1970 's. 

6 . Management Branch 

This branch is the most difficult branch to assess 
as it has the most complex structure. In order to more 
clearly discuss this branch, it was decided to illustrate 
the structural relations as was done for reliability 
management (Figure 33) . Figure 34 shows the five main 
subbranches, (1) Organization and Management, (2) Logistics, 
(3) Availability, (4) Procurement, and (5) Cost. Organiza- 
tion and Management is split into the two main twigs of 
Planning and Implementation and Evaluation and Assessment. 

The Procurement subbranch has developed two twigs. Contracts 
and Specifications. All subbranches showed growth patterns 
in the 1960 - 1970 decade. Organization and Management's 
growth can be traced to the military's desire to formalize 
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Figure 33 M Management Branch Structure 
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maintainability into a firm requirement. This was done 
when MIL-STD-470 and a number of supporting standards were 
issued. The curves in Figure 35 indicate that Program 
Planning and Implementation appeared to grow into the 
1970 's and shows a tendency to level off. ^Evaluation and 
Assessment techniques seemed to grow for a short period 
of time and then mature in the late 1960 's. Logistics 
concerns grew dramatically in the 1960 ’s as DOD put 
increased emphasis on operation and support costs. Interest 
in the amount of maintainability in Logistics appeared to 
decline in the 1970 's. This indicates that usage of main- 
tainability techniques in Logistics has matured. Availability 
does not show any dramatic increase other than from mid 
1960 's to early 1970 's. This increase related to the aware- 
ness that operational Availability deals not only with 
failure (reliability), but also with repair (maintainability). 

In this time frame, many articles in the literature stressed 
ideas on how to improve availability by considering main- 
tainability in addition to reliability. However, availa- 
bility appears to have experienced little or no growth in 
the 1970' s except in the past two years. Procurement's 
rise through the 1960 's is traced to specification of 
maintainability requirements in military contracts. The 
need to specify quantitive requirements forced more consider- 
ation of maintainability in specifications. 

7 . Maintainability Branch Correlation with General Applications 
Correlation of the most active main branches of 
maintainability with general applications indicates a strong 
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influence of maintainability interest within the Department 
of Defense and a lesser influence by NASA. The data base 
does not show a large interest in maintainability in con- 
sumables, construction, or industrial short-life equipment. 
The literature indicates a concern for maintainability by 
those companies manufacturing equipments which they sell 
or lease but continue to maintain throughout the equipment's 
service life. Computer and copying machine manufacturers 
are examples. The thrust for consumables was more towards 
reliability and quality control. Maintainability interest 
with respect to industrial long-life equipment appears to 
have been a concern in connection with commercial aircraft. 
Here again, the reasons appeared to be economics and cus- 
tomer satisfaction, since the airlines perform their own 
maintenance . 

The heavy influence of the Department of Defense 
is dramatically portrayed in Figures 36 and 37. Its in- 
terest appeared in the late 1950 's, rose sharply through 
the 1960 's, then appeared to decrease just as sharply in 
the late 1960's and early 1970's to the levels noted in the 
1950 's which it has maintained through the present day. 

NASA interest in maintainability has been primarily 
in space and its environments. The literature has been 
sparse with respect to number of published articles. 

Figures 36 and 37 show that awareness of maintainability by 
NASA did not really take hold until the middle 1960 's and 
it peaked in the late 1960 's when the space program design 
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was at its peak. Maintainability interest in the 1970's 
can be expected to increase with the skylab and space 
shuttle programs. 

In conclusion, the greatest thrust for maintaina- 
bility has come from the Department of Defense. 

F. RELIABILITY AND MAINTAINABILITY GROWTH COMPARISON 

The preceding sections in this Chapter have attempted 
to examine the growth of the reliability and maintainability 
disciplines by developing growth curves of the various 
branches within each discipline. The background discussions 
in Chapter II provided some insight into the events which 
affected the growth curves discussed earlier in this Chapter. 
Figure 38 provides an overview of the disciplines and 
summarizes how the emphasis has shifted within each discipline 
over the past fifteen years. 

Figure 38 indicates that reliability and maintainability 
main branch growths have progressed differently in any given 
time period. Reliability had very high emphasis on Manage- 
ment and Test 8 Evaluation in the late 1950 *s and early 
1960's. However, by 1965 the emphasis had started shifting 
more to Analysis and Design and this trend has continued 
to the present. Maintainability, on the other hand, had 
very high emphasis on design in the early 1960’s, but 
emphasis has subsequently shifted to Management and Test 8 
Evaluation . 

These trends correlate well with the events noted in 
Chapter II where, for example, it was noted that some of 
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the concepts developed for quality control were initially 
applied to reliability considerations during the 1950 *s. 
Quality control has a very strong management flavor and 
this is reflected in the high management emphasis of 
reliability illustrated in Figure 38 for 1960. This area 
and other aspects of management have received proportionately 
less emphasis (with respect to the other branches) in the 
1970’s. The late 1950's and early 1960*s saw an increase 
in study groups and organizations (AGREE and ARINC, for 
example) primarily devoted to analysis of systems and 
components and their influence is reflected in the dramatic 
increase in emphasis of Analysis (Reliability) in the mid 
1960’s. Design (Reliability) has also gradually received 
increased emphasis over the years, and this has led to 
development of specification and reliability design hand- 
books, particularly for DOD and .NASA. 

The initial strong Design emphasis in maintainability 
also produced a large number of specifications and design 
handbooks. However, as this discipline has grown, the 
flavor has shifted to Management and Test 8 Evaluation. 

Test 8 Evaluation (Maintainability) , particularly in the 
form of demonstrations to satisfy procurement specifications, 
appears to be steadily growing in emphasis, and it is 
expected that this trend will continue. 
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V. CONCLUSIONS AND RECOMMENDATIONS 



A. CONCLUSIONS 

Several conclusions regarding present and future em- 
phasis of reliability and maintainability have been reached 
as a result of the analysis of the growth trends of the 
disciplines. As noted earlier, DOD and NASA have strongly 
influenced the development of both Reliability and Main- 
tainability. Current trends, however, indicate that the 
energy (power) and the other service industries will play 
a major role in influencing the growth trends in the future 
particularly for Reliability. Maintainability growth will 
probably continue to be dominated by DOD because of its 
growing concern with the magnitude of operations and main- 
tenance cost of weapon systems. There seems to be growing 
concern in DOD and the defense industry that the present 
specifications for reliability and maintainability required 
in most weapon systems contracts are outdated because of 
technological advances made in recent years. Reliability 
and Maintainability Committees of Industry Associations, 
such as EIA, are investigating updating these standards and 
specifications jointly with government agencies. Several 
of the maintainability branches, notably Theory, Data, and 
Design are approaching maturity since the literature indicates 
almost no activity in these areas in recent years. 

Analysis is showing signs of continuing growth, although 
maturity may be reached in the near future. Two twigs in this 
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branch, Simulation/Modeling and MEA are indicating strong 
growth trends and it appears that this will continue. 

The Management and Test and Evaluation branches also 
show signs o£ continuing growth. Two twigs in the Test 
and Evaluation branch, Demonstration Methods and Reporting/ 
Evaluation, are also exhibiting strong growth patterns at 
present . 

Should the maintainability specifications be revised as 
proposed, then the Management and Test and Evaluation branches 
should receive renewed interest. 

The emphasis within the reliability discipline in recent 
years has shifted from the Management and Test and Evaluation 
branches to the Analysis and Design branches. Nevertheless, 
all of the main branches of Reliability are exhibiting 
healthy growth patterns at present, with Analysis and 
Management indicating increased activity in the last two 
years. This increased growth rate should continue under the 
impetus of renewed DOD interest in Management, and especially 
the newly elicited interest of ERDA and the service industries 
in both Management and Analysis. Applications of the tech- 
niques developed in the Analysis and Design branches to 
computational and power generation problems appear to be 
fast growing areas at present. This is particularly true 
of Power Generation which has established a phenomenal 
growth rate in the last three years and progress in this 
area should be very interesting to monitor in the future. 
Several twigs have emerged which show promise of high con- 
tinued growth in the future. These twigs, in conjunction 



TWIG 



MOTIVATION 



1. System Effectiveness 
(cost) 

2. Prediction 

3. Modeling/Simulation 

4. Warranties § 
Specifications 

5. Product Liability 






DOD and Service 



Consumer/Legal 



The twig, Product Liability, is potentially a very 
complex area with safety and reliability issues combined 
with the complicated legal implications involved with 
consumer protection. This twig is experiencing a sub- 
stantial growth rate. These issues are important because 
court decisions could have significant effects on the 
amount of Analysis and Testing a manufacturer may have to 
undertake in order to demonstrate that his product is re- 
liable and to precisely define the environmental rating. 

It appears that the literature in recent years has been 
devoting progressively more attention to the implications of 
equipment reliability on safety issues. Analysis tech- 
niques such as Fault Trees have been developed which are 
useful in both reliability and safety analysis. Many 
articles have dealt with these techniques as utilized in 
safety analysis as well as the effect of ''unreliability'' 
on safety. This area has been noticeably growing in the 
recent past and appears to have potential for rapid develop- 
ment in the immediate future. 
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B. RECOMMENDATIONS FOR FUTURE EFFORT 



1. Continue to update the data base with latest symposia 
proceedings. This effort has produced an extensive data 
base for the reliability and maintainability disciplines 
classified by keywords along the dimensions discussed in 
Chapter III. This data base, if kept current, represents 
a valuable source of information which is easily retrievable 
for research purposes. 

2-. Publish a bibliography on reliability and maintaina- 
bility. The data base on reliability and maintainability 
should be made available to other persons working in the 
disciplines . 

3. Develop the evolution of safety over the decades. 

The safety discipline is closely interrelated with reliability 
and maintainability and if developed would be a valuable 
source of information to future researchers. 

4. Present the taxonomies to symposia and professional 
societies for acceptance as standard classification systems. 
The taxonomies are composed of hierarchical ordered key- 
words whose structural relationships have been documented. 

The structural relationships are considered to be superior 
to those presently in existence. 

5. Expand the data base to include technical reports 
and classified documents. This will tend to fill in voids 
left by the current data base and provide a more complete 
analysis of the trends established herein. 



100 



APPENDIX A 



TAXONOMY KEYWORD GLOSSARY 
A.O. INTRODUCTION 

This glossary of taxonomy keywords is presented to 
provide the reader with the keyword definitions used by 
the authors to insure consistency in article keyword 
assignment. The glossary addresses only those taxonomy 
keywords about which there was some definition disagreement 
amongst the authors, or which was required for reader 
clarification. The glossary is not intended to be a dic- 
tionary of reliability and maintainability terms for 
general use. 

The keyword definitions, where possible, have been 
taken from the literature when it appeared that the 
definition had general usage. Definitions were generated 
for keywords that did not have generally accepted definitions 
and these definitions have been identified by asterisks. 
Keywords which have generally accepted definitions such as 
Mechanical or Communications, and keywords which are 
taxonomy headings, such as Management and Prediction, have 
not been defined. Where needed for clarity, the taxonomy 
branch to which the keyword being defined belongs has been 
parenthetically inserted following the keyword. 

A.l. DEFINITIONS 

AEC/ERDA - articles which deal with energy either sponsored 
by or written for the Atomic Energy Commission or the Energy 
Resource Development Agency. 



* Analysis and Evaluation - (Test 5 Evaluation) process of 
working with the reported test results to arrive at con- 
clusions relative to the item tested. 

Apportionment - an allocation of the overall numerical 
reliability or maintainability requirements among each of 
the elements of a system. 

Attribute/Parameter Estimation - (Test 5 Evaluation) - 
measurement and/or correlation of some characteristic of 
a part or system by test procedures wherein the character- 
istic is quantitatively or qualitatively estimated by 
analyzing the test data to determine if the part/system 
possesses the characteristic. 

Availability - the probability that the system is operating 
satisfactorily or is ready to be placed in operation at 
any point in time under stated conditions of use. 

* Bayesian - technique of modifying conclusions reached 
about present data based on historical knowledge of the 
data characteristics. 

* Block Diagram - a graphical method relating system success 

•to component successes. 

Burn- in - the operation of an item, component or system 
to stabilize its characteristics or to weed out its faults 
prior to placing the item in use. 

* Case Studies - how specific reliability or maintainability 
programs were approached and/or how specific program 
problems were resolved. The essence of these articles is 
not analytical but rather lessons learned for discussions 
of "How we did what." 

* Component Testing (Design) - subjecting components or 
systems to environments designed to verify that they will 
perform their intended function in the environments in 
which they will be utilized. 

* Consumables - items such as ammunition, food and fuel which 
are expended during operation of the system. 

^ Consumer Capital Item - items such as automobiles and 
appliances, which by virtue of their cost and life duration 
represent a significant portion of the average citizen's 
budget . 

* Cost - the costs, monetary or otherwise, of a reliability 
or maintainability program, including articles concerning 
major programs or concepts, such as design- to-cost , 
operation and maintenance costs, and their relationship to 
reliability and maintainability. 
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* Criteria/Requirements (Design) - goals and specified 
performance values which components and/or systems are 
expected to achieve in their operational environment. 

Criticality Analysis - a quantitative method of ranking 
the failure modes according to the criticality of their 
effects to their probability of occurrence. 

* Data Collection - methods, forms, practices, etc., for 
the acquisition and storage of data concerning the per- 
formance of specified entities. 

* Data Exchange - programs for the exchange of data v;ithin 
the reliability and maintainability communities. The 
definition includes articles presenting bibliographies 
developed to facilitate the exchange. 

* Data Utilization - purposes for which information con- 
cerning systems or component performance is to be 
collected. 

Design Disclosure Format (Data) - a method of presenting 
critical maintainability data for the system under design 
in a manner that facilitates analysis. 

* Design of Experiments (Test § Evaluation) - development 
of test procedures/methods in such a way that statistical 
principles may be utilized in working with the test data 
and in reaching conclusions. 

* Design Reviews (Analysis) - a comprehensive critical audit 
of all aspects of the design of the hardware and software. 

^ Design Reviews (Management) - management reviews of 
reliability and maintainability programs. These reviews 
tend to be macro program reviews vice the more micro 
design reviews identified under analysis. 

* DOD - articles dealing with defense applications sponsored 
by or written for or about the Department of Defense. 

’^ Failure Recurrence Control - articles concerning management 
efforts to curtail and/or preclude the recurrence of 
reliability failures. Includes articles addressing re- 
liability failures which arise during or as a result of 
development or production. 

Fault Tree - a graphical method using logic calculus to 
relate system failures to component failures or failure 
modes . 

’^ Field Activity - end user conditions caused by inadequate 
reliability or maintainability, and efforts to improve or 
rectify the situation. 
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FMEA - a qualitative technique for analyzing and evaluating 
a design by singling out failure modes and analyzing the 
consequences and causes. 

* General (Functional Area) - articles which do not address 
a functional area, or articles where the functional area 
though addressed is immaterial to the essence of the 
article . 

* Goodness of Fit - technique to determine if a group of 
data can be expected to behave in accordance with a hy- 
pothesized probability distribution at some predetermined 
level of significance. 

* Government (Other) - articles dealing with applications 
sponsored by government agencies other than AEC/ERDA, 

NASA, or DOD. This includes foreign governments. 

* High Reliability Parts (Design) - those parts or compo- 
nents which by virtue of design, construction or testing 
are more resistant to failure than parts with similar 
form, fit and function. 

Human Engineering - the area of human factors which applies 
scientific knowledge to the design of items to achieve 
effective man-machine integration and utilization. 

* Industrial Long Life or High Cost - items such as heavy 
machine tools which require a significant portion of an 
industrial concern’s capital budget. 

* Industrial Short Life or Low Cost - items , such as test 
equipment, utilized for industrial purposes which represent 
expensed rather than capitalized expenditures or which 
represent a small portion of an industrial concern's budget. 

Logistics - the procurement, movement, maintenance and 
disposition of supplies, equipment, facilities and personnel, 
and the provision of services. 

MEA - the formal process of reviewing a design or proposed 
design for the purpose of identifying and quantifying the 
maintenance and support requirements for an operational 
system. 

Measure - the use of theory which allows quantification of 
maintainability parameters so they can be measured. 

Modeling - conceptualizing the interactions of the elements 
comprising an entity in quantifiable terms usually for the 
purposes of behavior prediction and/or forming hypotheses 
about current or past behavior. 
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* NASA - articles which deal with space applications sponsored 
by or written for or about the National Aeronautics and 
Space Administration. 

* Non- reversible - devices which depend upon some physical, 
chemical or biological reaction or effect which once 
started cannot be reversed or changed back to its original 
form or state. 

^ Parameter Prediction - techniques for estimating the 
characteristics of a population based on the knoivledge of 
the behavior of a sample drawn from the population or a 
knowledge of its constituent parts, functions and operating 
environment . 

* Part/Material Selection - selection of those parts which, 
by design or from test results, are more failure resistant 
than similar parts. 

Probability Distribution - a mathematical description of 
the relative frequency with which the various outcomes of 
an experiment can be expected to occur. 

^ Product Liability - implied responsibility of manufacturer 
to ensure that tneir products perform their mission safely 
and satisfactorily. Consequences of product failures may 
be found to be responsibility of product manufacturer. 

* Pro^ram Evaluation (Management) - assessment and interpre- 
tation of reliability or maintainability programs. 

* Qualitative Attributes - those techniques such as access- 
ibility , test points, that allow ease of maintenance. 

* Quant itative Attributes - those characteristics that can 
be assessed and measured or demonstrated during established 
design, development, production, and field use milestones. 

Redundancy (Reliability, Design) - the existence of more 
than one path for accomplishing a given task, where all 
means must fail before there is an over-all failure of 
the system. 

* Reliability Groivth - the process of predicting future 
reliability achievements by extrapolating measured data 
on current reliability characteristics. 

Renewal Theory - the study of certain probability problems 
which can arise in connection with the failure and replace- 
ment of components. 

Safety Factor - the margin of conservativeness inherent in 
the application of component parts as a function of all 
stresses. Includes using components in environments less 
stressful then normally designed for (derating) . 
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Sampling/Screening Plan - a statement of the sample size or 
sizes to be used and the associated acceptance and rejection 
criteria. 

* Service - articles sponsored by or written for or about the 
service industries, (ie. utilities, airlines, telephone). 

* Simulation/Modeling - conceptualizing the interactions of 
the elements comprising an entity in quantifiable terms, 
usually for the purposes of behavior prediction and/or 
forming hypotheses about current or past behavior. 

Specifications (Procurement) - considerations related to 
specifications or the statement of work. 

System Effectiveness (Cost) - the measure of the extent to 
wnich a system can be expected to complete its assigned 
mission within an established time frame under stated 
environmental conditions. 

* Test Equipment - equipment utilized to subject components 
or systems to conditions designed to verify the item's 
performance under speci^fied conditions. 

* Testing/Demonstration Methods - techniques to measure 
whether systems, components, etc., will perform at specified 
or required levels. 

Tolerance Analysis - study of the allowable variation in 
values of a component or system characteristics within which 
the characteristics are judged acceptable. 

Trade-off - a process of optimization of the related factors 
involved during the life cycle of an equipment or system 
to determine an optimal balance among these factors to 
accomplish a specified objective. 

* Unspecif ied (Application, General) - application unspecified 
in the article . 
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